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Abstract. We consider the problem of symbolic reachability analysis of 
higher-order context-free processes. These models are generalizations of 
the context-free processes (also called BPA processes) where each process 
manipulates a data structure which can be seen as a nested stack of 
stacks. Our main result is that, for any higher-order context-free process, 
the set of all predecessors of a given regular set of configurations is regular 
and effectively constructible. This result generalizes the analogous result 
which is known for level 1 context-free processes. We show that this 
result holds also in the case of backward reachability analysis under a 
regular constraint on configurations. As a corollary, we obtain a symbolic 
model checking algorithm for the temporal logic E(U,X) with regular 
atomic predicates, i.e., the fragment of CTL restricted to the EU and EX 
modalities. 



1 Introduction 

Pushdown systems and their related decision and algorithmic analysis problems 
(reachability analysis, model checking, games solving and control synthesis, etc) 
have been widely investigated in the last few years [11I7I22I5I15I8I2| . This recent 
intensive research effort is mainly motivated by the fact that pushdown sys- 
tems arc quite natural models for sequential programs with recursive procedure 
calls (see e.g., |16ll4j ). and therefore they are particularly relevant for software 
verification and design. 

Higher-order pushdown systems [13] (HPDS) are generalizations of these 
models in which the elements appearing in a pushdown stack are no longer 
single letters but stacks themselves. We call this kind of nested stack structures 
higher- order stores. Stores of level 1 are sequences of symbols in some finite 
alphabet (those are standard pushdown stacks), and stores of level n + 1 are 
sequences of stores of level n, for any n > 1. The operations allowed on these 
structures are (1) the usual push and pop operations on the top-most level 1 
store, (2) higher-order push and pop operations allowing to duplicate or erase 
the top-most level k store of any given level k < n. 

This general model is quite powerful and has nice structural characterizations 
[12110] . It has been in particular proved in [T§] that HPDS are equivalent to 
(safe) higher-order recursive program schemes. Interestingly, it has also been 
proved that the monadic second-order theory of an infinite tree generated by a 



HPDS is decidable [19111] , which generalizes the analogous result for pushdown 
systems proved by Muller and Schupp [5D]. Also, it has been proved that parity 
games can be solved for HPDS [5], which generalizes the result of Walukiewicz 
for pushdown systems [55]. These results actually show that model checking is 
decidable for HPDS. However, they only allow to check that a property holds in 
a single initial configuration and they do not provide a procedure for computing 
a representation of the set of configurations which satisfy some given property 
(the satisfiability set of the property). 

The basic step toward defining an algorithm which effectively computes the 
satisfiability sets of properties is to provide a procedure for computing the set of 
backward reachable configurations from a given set of configurations, i.e. their 
set of predecessors. In fact, the computation of forward- or backward-reachable 
sets is a fundamental problem in program analysis and in verification. 

Since HPDS are infinite-state systems, to solve this problem we need to con- 
sider symbolic representation structures which (1) provide finite representations 
of potentially infinite sets of configurations, and (2) enjoy closure properties and 
decidability properties which are necessary for their use in verification. Mini- 
mal requirements in this regard are closure under union and intersection, and 
decidability of the emptiness and inclusion problems. 

A natural class of symbolic representations for infinite-state systems is the 
class of finite-state automata. Recently, many works (including several papers 
on the so-called regular model- checking) have shown that finite-state automata 
are suitably generic representation structures, which allow to uniformly handle 
a wide variety of systems including pushdown systems, FIFO-channel systems, 
parameterized networks of processes, counter systems, etc. [51311811123161411?] . 

In particular, for the class of pushdown systems, automata-based symbolic 
reachability analysis techniques have been developed and successfully applied in 
the context of program analysis |5|15I21| . Our aim in this paper is to extend this 
approach to a subclass of HPDS called higher-order context-free processes (HCFP 
for short). This class corresponds to the higher order extension of the well-known 
context-free processes (also called BPA processes). HCFP can actually be seen as 
HPDS with a single control state, similarly to level 1 CFP which are equivalent 
to level 1 PDS with a single control state. The contributions of our paper can 
be summarized as follows. 

First, we observe that, due to the duplication operation, the set of immediate 
successors (i.e. the post image) of a given regular set of configurations is in general 
not regular, but it is always a context-sensitive set. 

Then, we prove that, and this is our main result, for every HCFP of any 
level, the set of all predecessors (i.e. the pre* image) of any given regular set 
of configurations is a regular set and effectively constructible. As a corollary of 
this result, we obtain a symbolic model checking algorithm (an algorithm which 
computes the set of all configurations satisfying a formula) for the temporal 
logic E(F,X) with regular atomic predicates, i.e., the fragment of CTL with the 
modalities EF (there exist path where a property eventually holds) and EX (there 
exist an immediate successor satisfying some property). 



Furthermore, we extend our construction of the pre* images by showing that 
the set of predecessors under a regular constraint (i.e., the set of all predecessors 
reachable by computations which stay in some given regular set of configurations) 
is also regular and effectively constructible. For that, we use representation struc- 
tures which can be seen as alternating finite-state automata. This result allows 
us to provide a symbolic model checking algorithm for the logic E(U, X) with reg- 
ular atomic predicates, i.e., the fragment of CTL with the operators EU (exists 
until) and EX (exists next). 

The structure of this paper is the following. In the next two sections, we 
introduce higher-order stores and the model of higher-order context-free pro- 
cesses. We also provide a symbolic representation for (infinite) regular sets of 
stores using a certain type of finite automata. Then, for the sake of readability, 
we first present our algorithm for computing the unconstrained pre and pre* 
sets of a regular set of stores (Section 2]), before extending it to the case of pre* 
sets constrained by a regular set C (Section [5]). Due to lack of space, additional 
definitions and detailed proofs can be found in the full version of this papeiQ. 

2 Higher-order Context-free Processes 

We introduce a class of models we call higher- order context-free processes, which 
generalize context-free processes (CFP) and are a subclass of higher-order push- 
down systems (HPDS). They manipulate data structures called higher-order 
stores. 

Definition 2.1 (Higher-order store). The set Si of level 1 stores (or 1- 
storesj over store alphabet r is the set of all sequences [a\ . . . ai] <G [P*]. For 
n > 2, the set S n of level n stores (or n-storesj over r is the set of all sequences 
[ Sl ...si] 6 [<S n _!+]. 

The following operations are defined on 1-stores: 

push™([ai . . . ai]) = [wa2 ... a;] for all w G r*. 

top^ai . . .a{\) = ai. 

We will sometimes abbreviate push\ as pop x . The following operations are de- 
fined on n-stores (n > 1): 



push™([si . 


■si}) = 


[p?/s/l5"(si) ...Si] 




push k ([s! . 


■si}) = 


[push k (s!) . . . Sl] 


if k G [2,n[, 


push n ([s! . 


■*l]) = 


[sisi ■■■si] 




pop k {[si . 


■*l]) = 


\pop k (s 1 )...si] 


if k G [2,n[, 


pop n ([si . 


■*l]) = 


[s 2 ■ ■ ■ Sl] 


if I > 1, else undefined, 


top k ([s! . 


■si}) = 


top k (si) 


if k G 


top n {[si . 


■si]) = 


Sl- 





available at http: //www. liaf a. jussieu. f r/~{}ameyer/ 



We denote by O n the set of operations consisting of: 



{ push k , pop k | fc G [2, n] } U { push™ | w G -T* }. 



We say that operation o is of level n, written l(o) = n, if o is either push n or 
pop n , or push™ if n = 1. We can now define the model studied in this paper. 

Definition 2.2. A higher-order context-free process of level n (or n-HCFP) is 
a pair TL = (r, A), where r is a finite alphabet and A G F x O n is a finite set 
of transitions. A configuration of TL is a n-store over r. TL defines a transition 
relation c — > between n-stores (or > when TL is clear from the context), where 



The level 1(d) of a transition d — (a, o) is simply the level of o. Let us give 
a few more notations concerning HCFP computations. Let H ~ (r, A) be a 
n-HCFP. A run of TL starting from some store Sq is a sequence S0S1S2 ■ ■ ■ such 
that for all i > 0, Si <^-> s^+i. The reflexive and transitive closure of is written 
> and called the reachability relation. For a given set C of n-stores, wc also 
define the constrained transition relation = <— > H (CxC), and its reflexive 
and transitive closure ^->c- Now for any set of n-stores S, we consider the sets: 



When C is the set S n of all n-storcs, we omit it in notations and simply write 
for instance pre n (S) instead of pre n [C](S) . We will also omit TL when it is clear 
from the context. When TL consists of a single transition d, we may write pre d (S) 
instead of pre n (S). 

3 Sets of Stores and Symbolic Representation 

To be able to design symbolic verification techniques over higher-order context- 
free processes, we need a way to finitely represent infinite sets (or languages) of 
configurations. In this section we present the sets of configurations (i.e. sets of 
stores) we consider, as well as the family of automata which recognize them. 

A n-store s = [si . . . si] over r is associated to a word w(s) = [w(si) . . . w(si)), 
in which store letters in _T only appear at nesting depth n. A set of stores over _T 
is called regular if its set of associated words is accepted by a finite automaton 
over r' = ru{[,]}, which in this case we call a store automaton. We will often 
make no distinction between a store s and its associated word w(s). Due to the 
nested structure of pushdown stores, it will sometimes be more convenient to 
characterize sets of stores using nested store automata. 



H 




3(a, o) G A such that topi(s) = a and s = o(s). 



post H [C](S) 

post* H [C](S) 
pre H [C]{S) 

pre* n [C](S) 



{s | 3s' G S, s' ^ c s}, 

{s\3s'eS, s'^cs}, 
{s\3s'eS, s^ c s'}, 

{s | 3s' G S, s^c s'}. 



Definition 3.1. A level 1 nested store automaton is a finite automaton whose 
transitions have labels in P. A nested store automaton of level n > 2 is a finite 
automaton whose transitions are labelled by level n — 1 nested automata over F. 

The existence of a transition labelled by B between two control states p and q 

B B 

in a finite automaton A is written p — > q, or simply p — > q when A is clear 

A 

from the context. Let A — (Q,r,5,qo,qf) be a level n nested automatorQ with 
n > 2. The level k language of A for k e [l,n] is defined recursively as: 

L k (A) = { [L k (Ax) . . . L k (Ai)} | [Ai... Ai] E L„ (A) } if k < n, 

L k (A) = { [At .. .Ai] | q ^ . .. q f } if k = n. 

For simplicity, we often abbreviate L\(A) as L(A). We say a nested automaton 
B occurs in A if B labels a transition of A, or occurs in the label of one. Level 
n automata are well suited to representing sets of n-stores, but have the same 
expressive power as standard level 1 store automata. 

Proposition 3.2. The store languages accepted by nested store automata are 
the regular store languages. 

Moreover, regular n-store languages are closed under union, intersection and 
complement in S n . We define for later use the set of automata { A™ \ a E T, n E 
N} such that for all a and n, L(A%) = { s E S n | top^s) = a}. We also write 
Ax B the product operation over automata such that L{A x B) = L(A) n L(B). 

4 Symbolic Reachability Analysis 

Our goal in this section is to investigate effective techniques to compute the sets 
pre(S), post(S), pre*(S) and post*(S) for a given n-HCFP H, in the case where 

5 is a regular set of stores. For level 1 pushdown systems, it is a well-known 
result that both pre^AS) and post^S) are regular. We will see that this is still 
the case for pre(S) and pre*{S) in the higher-order case, but not for post(S) 
(hence not for post*(S) either). 

4.1 Forward Reachability 

Proposition 4.1. Given a n-HCFP TL and a regular set of n- stores S, the set 
post(S) is in general not regular. This set is a context-sensitive language. 

Proof. Let post^ a o j(S) denote the set { s' \ 3s E S, top 1 (s) = a A s' = o(s) }. 
Suppose S is a regular set of n-stores, then if d ~ {a,push™) or d ~ {a,pop k ), it 
is not difficult to see that post^ a ^(S 1 ) is regular. However, if d = (a,push k ) with 
k > 1, then post^ a o ^(S) is the set { [ n ~ k+1 ttw \ [ n ~ k+1 tw E S }. ft can be shown 
using the usual pumping arguments that this set is not regular, because of the 
duplication of t. However, one can straightforwardly build a linearly bounded 
Turing machine recognizing this set. □ 

2 Note that we only consider automata with a single final state. 



4.2 Backward Reachability 



We first propose a transformation on automata which corresponds to the pre 
operation on their language. In a second time, we extend this construction to 
deal with the more difficult computation of pre* sets. 

Proposition 4.2. Given a n-HCFP TL and a regular set of n-stores S, the set 
pre(S) is regular and effectively computable. 

We introduce a construction which, for a given HCFP transition d and a given 
regular set of n-stores S recognized by a level n nested automaton A, allows us to 
compute a nested automaton A' d recognizing the set pre(S) of direct predecessors 
of S by d. This construction is a transformation over nested automata, which 
we call T d . We define A' d = T d {A) = (Q', T, 5', q' , q f ) as follows. 
If 1(d) < n, we propagate the transformation to the first level n — 1 automaton 
encountered along each path. We thus have Q' = Q, q' Q = qo and 

o = {q — > qi | qo —* qi\\J{q > q \ q — + q A q ^ q }. 

If 1(d) = n, we distinguish three cases according to the nature of d: 

1. If d= (a, push™), then Q' = Q U {q' } and 5' = S U { q' Q —* q\ \ qo — > q% }. 

2. If d = (a,push n ) and n > 1, then Q' = Q U {<?o} and 

5' = S U { q' Q <?2 | g 9i 92 } where B = Ai x A 2 * A a n ~ l) ■ 

3. If d = (a, pop n ), then Q' = Q U {g } and <5' = <5 U { q - — ► go }• 

It is not difficult to prove that L(A' d ) = pre d (L(A)). Hence, if A is the set of 
transitions of Tt, then we have pre(S) = pre(L(A)) = Udezi ^(Aj). 

This technique can be extended to compute the set pre* (S) of all predecessors 
of a regular set of stores S. 

Theorem 4.3. Given a n-HCFP H and a regular set of n-stores S , the set 
pre*(S) is regular and effectively computable. 

To compute pre*(S), we have to deal with the problem of termination. A 
simple iteration of our previous construction will in general not terminate, as 
each step would add control states to the automaton. As a matter of fact, even 
the sequence (pre l (S))t>o, defined as pre°(S) = S and for all n > 1 pre n (S) = 
pre n ~ 1 (S) U pre(pre n ^ 1 (S)), does not reach a fix-point in general. For instance, 
if d = (a,popi), then for all n, pre n ([a\) = { [a 1 ] \ i < n } ^ pre n+1 ([a\). 

To build pre*(S) for some regular S, we modify the previous construction 
in order to keep constant the number of states in the nested automaton we 
manipulate. The idea, instead of creating new control states, is to add edges 
to the automaton until saturation, eventually creating loops to represent at 
once multiple applications of a HCFP transition. Then, we prove that this new 
algorithm terminates and is correct. 



Let us first define operation Td for any n-HCFP transition d (see Figure Q] 
for an illustration). Let A = (Q, T, 8, q , qj) and A = (Q, T, 5', go, <?/) be nested 
n-store automata over r = TU {[,]}, and d a n-HCFP transition. We define 
A = T d {A) as follows. 

If the level of d is less than n, then we simply propagate the transformation to 
the first level n — 1 automaton encountered along each path: 

8 = {qo — > gi <7o — ► 3i } U { g — > 9 9 — ^A?^ }- 

.A *4 

If Z(d) = n then as previously we distinguish three cases according to d: 

1. If n = 1 and d = (a, pushT), then <5' = (5 U { go — ► <7i I Qo <7i }• 

A 

2. If d = (a,push n ) for some n > 1, then 

5' = 8 U {<?o g 2 | g ^ <?i — ^ <?2 } where B = Ai x A 2 x A^ l ~ 1 ' ) . 

A A 

3. If d = (a, pop n ), then <5' = (5 U { qo ™ — ► <7o } 

Suppose 77 = (r, A) with Z\ = { do, . . . , d;_i }. Given an automaton A such that 
S = L(A), consider the sequence (Ai)i>o defined as Aq — A and for allz > and 
j = i mod Ai+i = Td^Ai). In order to obtain the result, we have to prove 
that this sequence always reaches a fix-point (Lemma 14. 4[) and this fix-point is 
an automaton actually recognizing pre*(S) (Lemmas 14.51 and I4.6|) . 



_ A 



A 



B 



Fig. 1. transformation Td(A) for d = (a, push™), (a,push k ) and (a,pop k ). 



Lemma 4.4 (Termination). For cd/ nested n-store automaton A and n-HCFP 
Ti, = (T,A), the sequence (Ai)i>o defined with respect to A eventually stabilizes: 
3k > 0, Vfc' 6 A, Ak' = Ak, which implies L(Ak) = Ui>o L(Ai). 

Proof. First, notice that for all d, Td does not change the set of control states of 
any automaton occurring in A, and only adds transitions. This means (-4i)i>o 
is monotonous in the size of each Ai . 

To establish the termination of the conctruction, we prove that the number of 
transitions which can be added to Aq is finite. Note that by definition of Td, the 
number of states of each Ai is constant. Moreover, each new transition originates 
from the initial state of the automaton it is added to. Hence, the total number 
of transitions which can be added to a given automaton is equal to \V n \ ■ \Q\, 
where V n is the level n vocabulary and Q its set of states. Since \Q\ does not 



change, we only have to prove that V n is finite for all n. If n = 1, V\ = T, 
and the property holds. Now suppose n > 1 and the property holds up to level 
ii—l. By induction hypothesis, V n -i is finite. With this set of labels, one can 
build a finite number TV of different level n — 1 automata which is exponential 
in |V^_i| • K, where K depends on the number of level n — 1 automata in Aq 
and of their sets of control states. As each transition of a level n automaton is 
labelled by a product of level n — 1 automata, then | V n | is itself exponential in 
N, and thus doubly exponential in |V^_i|. Remark that, as a consequence, the 
number of steps of the construction is non-elementary in n. □ 

Lemma 4.5 (Soundness). [J i>0 L(Ai) C pre^(S). 

Proof (sketch). We prove by induction on i the equivalent result that Vi, L(Ai) C 
pre^S). The base case is trivial since by definition Aq = A and L(A) = S C 
pre^t(S). For the inductive step, we consider a store s accepted by a run in Ai+\ 
and reason by induction on the number m of new level k transitions used in this 
run, where k is the level of the operation d such that Ai+i = Td(Ai). The idea 
is to decompose each run containing m new transitions into a first part with less 
than m new transitions, one new transition, and a second part also containing 
less than m new transitions. Then, by induction hypothesis on m and i, one can 
re-compose a path in Ai recognizing some store s' such that s' £ pre^ i {S) and 
s G preys'). □ 

Lemma 4.6 (Completeness). pre^(S) C Ui>o-^(-^)- 

Proof (sketch). We prove the sufficient property that for all nested store automa- 
ton A and HCFP transition d, pre d {L{A)) C L(Td(^4)). We consider automata ^4 
and .A' such that A' = Td(A), and any pair of stores s £ L(A) and s' s (s). 
It suffices to isolate a run in A recognizing s and enumerate the possible forms 
of s' with respect to s and ci to be able to exhibit a possible run in A' accepting 
s', by definition of Td- This establishes the fact that Td adds to the language L 
of its argument at least the set of direct predecessors of stores of L by d. □ 

As a direct consequence of Proposition 14.21 and Theorem 14. 3( we obtain a 
symbolic model checking algorithm for the logic E(F,X) with regular store lan- 
guages as atomic predicates, i.e. the fragment of the temporal logic CTL for the 
modal operators EF (there exists a path where eventually a property holds) and 
EX (there exist an immediate successor satisfying a property). 

Theorem 4.7. For every HCFP TL and formula ip of E(F, X), the set of config- 
urations (stores) satisfying tp is regular and effectively computable. 

5 Constraining Reachability 

In this section we address the more general problem of computing a finite au- 
tomaton recognizing pre^[C](S) for any HCFP H and pair of regular store 
languages C and S. We provide an extension of the construction of Proposition 



14.31 allowing us to ensure that we only consider runs of H whose configurations 
all belong to C . Again, from a given automaton A, we construct a sequence of 
automata whose limit recognizes exactly pre^ i [C](L(A)) . The main (and only) 
difference with the previous case is that we need to compute language inter- 
sections at each iteration without invalidating our termination arguments (i.e. 
without adding any new states to the original automaton). For this reason, we 
use a class of alternating automata, which we call constrained nested automata. 

Definition 5.1 (Constrained nested automata). Let B be a non-nested m- 
store automator^ (with m > n). A level n B- constrained nested automaton A 
is a nested automaton {QA,r,S A ,i Al f A ) with special transitions of the form 

p ~-^> (q, r) where p,q E Qa> r is a control state of B and C is a level n — 1 

B-constrained nested automaton. 

For lack of space, we are not able to provide here the complete semantics 
of these automata. However, the intuitive idea is quite simple. Suppose A is a 
S-constrained nested n-store automaton, and B also recognizes n-stores. First, 
we require all the words accepted by A to be also accepted by B: L(A) C L(B). 

Then, in any run of A where a transition of the form p — ► (q, r) occurs, the 
remaining part of the input word should be accepted both by A when resuming 
from state q and by B when starting from state r. Of course, when expanding T> 
into a word of its language, it may require additional checks in B. As a matter 
of fact, constrained nested automata can be transformed into equivalent level 1 
alternating automata. As such, the languages they accept are all regular. 

Proposition 5.2. Constrained nested automata accept regular languages. 

The construction we want to provide needs to refer to whole sets of paths 
in a level 1 store automaton recognizing the constraint language. To do this, we 
need to introduce a couple of additional definitions and notations. 

Definition 5.3. Let A be a finite store automaton over T' = ru{ [, ] }. A state 
p of A is of level if it has no successor by [ and no predecessor by ]. It is of 
level k if all its successors by [ and predecessors by ] are of level k — 1. The level 
of p is written l(p). 

We can show that any automaton recognizing only n-stores is equivalent to 
an automaton whose control states all have a well-defined level. A notion of level 
can also be defined for paths. A level n path in a store automaton is a path 
pi . . .pk with l{p\) = l{pk) = n and Vi G [2, k — 1], l{pi) < n. All such paths 
are labelled by n-stores. Now, to concisely refer to the whole set of level n paths 
between two level n control states, we introduce the following notation. Let 

Q = {qeQ A I l(q) < n A p x q p 2 } 

A A 



3 i.e. a standard, level 1 finite state automaton. 



be the set of all states of A occurring on a level n path between p\ and p 2 ■ If Q 

B 

is not empty, we write pi p 2l where B is defined as: 

A 

B= {Qb = QU{ Pl ,p 2 }, r', 5 B =S A r\(Q B x r' x Q B ), p x , pa). 
Thanks to these few notions, we can state our result: 

Theorem 5.4. Given a n-HCFP 7i and regular sets of n-stores S and C , the 
set pre^[C](S) is regular and effectively computable. 

To address this problem, we propose a modified version of the construc- 
tion of the previous section, which uses constrained nested automata. Let d = 
(a, 6) be a HCFP transition rule, A = (Qa, T,d,i, f) and A' = (Q^,r,d',i,f) 
two nested fc-store automata constrained by a level 1 n-storc automaton B = 
{Qb, r', Sjs, iBi /b) accepting C (with n > k). We define a transformation Tj 3 (A), 
which is very similar to T^. , except that we need to add alternating transitions 
to ensure that no new store is accepted by A' unless it is the transformation of 
a store previously accepted by B (Cf. Figure [2]). If 1(d) < k, we propagate the 
transformation to the first level k — 1 automaton along each path: 

6' = { l T ^ ) (p,q) | ^ (p,q) } U {p (p',q') eS\p^t}. 
A 

If 1(d) = n, we distinguish three cases according to the nature of d: 

1. If d = (a, push™), then 

6' = S U { i (p, q) | i (p, q') A 3qi,q € Qb, 

%i) = %) = 0, is qi <?}■ 

2. If d = (a,push k ), then for to = n—k+1 andC = (Ci xC 2 ) x (B\ xB 2 ) x .4i fc_1 \ 

6' = S U { i (p, q) I i (p,q') A 3qi, q 2 , q € Qb, 

Ai Ai 

%l) = %2) = %) = 1, »B 91 ^ 92 ^ <?}• 

3. If d = (a,pop k ), then for m = n — A; + 1, 

<5' = <S U { i (i,g) I 3qeQ B , /(g) = fc - 1, ig^ 9 }. 

Bi 

Suppose H = (r, A) with A = { do, ■ ■ ■ 1 c?/— 1 }■ Given an automaton A such that 
5 = L(A), consider the sequence (A)i>o defined as Ao = A B (the 23-constrained 
automaton with the same set of states and transitions as A, whose language is 
L(A) n L(B)) and for all % > and j = i mod I, A t+1 = Tj 5 (Ai). By definition 
of T®, the number of states in each A4 does not vary, and since the number of 



Fig. 2. transformation T®(A) for d = (a, push™), (a, push k ) and (a,pop k ). 

control states of B is finite the same termination arguments as in Lemma 14.41 
still hold. It is then quite straightforward to extend the proofs of Lemma l4~5l and 
Lemma 14751 to the constrained case. 

This more general construction also allows us to extend Theorem 14.71 to the 
larger fragment E(U,X) of CTL, where formulas can now contain the modal 
operator EU (there exists a path along which a first property continuously holds 
until a second property eventually holds) instead of just EF. 

Theorem 5.5. Given a HCFP TL and formula ip of E(U,X), the set of configu- 
rations (stores) satisfying ip is regular and effectively computable. 

6 Conclusion 

We have provided an automata-based symbolic technique for backward reach- 
ability analysis of higher-order context-free processes. This technique can be 
used to check temporal properties expressed in the logic E(U, X). In this respect, 
our results provide a first step toward developing symbolic techniques for the 
model-checking of higher-order context-free or pushdown processes. 

Several important questions remain open and arc left for future investigation. 
In particular, it would be interesting to extend our approach to the more general 
case of higher-order pushdown systems, i.e. by taking into account a set of control 
states. This does not seem to be technically trivial, and naive extensions of our 
construction lead to procedures which are not guaranteed to terminate. 

Another interesting issue is to generalize our symbolic approach to more 
general properties than reachability and/or safety, including liveness properties. 
Finally, it would also be very interesting to extend our symbolic techniques in 
order to solve games (such as safety and parity games) and to compute repre- 
sentations of the sets of all winning configurations for these games. 
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A Appendix 



A.l Nested store automata 

Proposition A.l. Nested store automata accept regular store languages. 

Proof. We will prove that given a nested store automaton A = (Q,r,5,i, f), one 
can effectively compute a level 1 store automaton A[ such that L\(A) = L{A[). 
We reason by induction on the level n of A. For n = 1, the property trivially 
holds. For greater values of n, consider the property as true for all levels less than 
n and let Ai . . . A m be the level n — 1 automata labelling the transitions of A. 
By induction hypothesis, we can build level 1 automata Ail ■ ■ ■ -Ami such that 
Vj G [l,m], Li(Aj) = L(Ajl). Let Aji= (Qj,r,8j,ij, /,), with all Qj supposed 
disjoint. Wc now build the level 1 automaton Ai— (Q' , r, 5', i', /') where for all 

v. a G Q, 7 G [1, ml, r, s, t, u € Q-i and n£f such that p — U o, ii — — > r, s — t 

and it — ► f,-, we have: 
Ail J3 ' 

■ I I I a ] j. ] j,, 

i — -v % p — -> pr ps pt pu — > g / — > / 

XI -At At At 

According to this construction, a path of „4J. between two control states p and 
5 in Q fl Q' is labelled by a word s if and only if s represents a (n — l)-store 

accepted by some Aj such that p —A q. Hence ^4J. accepts all words of the form 

[si ... si] such that [Ai 1 . . - Ai,} G L n (A) and for all j, Sj G L(Ai j ), which is 
precisely the definition of L(A). □ 

Before stating the converse, we need to introduce the notion of level of a 
level 1 store automaton control state. Let A be a finite store automaton over 
r' = r Li {[,]}. A state p of A is of level if it has no successor by [ and 
no predecessor by ]. It is of level k if all its successors by [ and predecessors 
by ] are of level k — 1. The level of p is written l(p). We also define a notion 
of level for paths. A level n path in a store automaton is a path pi . . .pk with 
Z(pi) = l(pk) = n and Vi G [2, k — 1], /(p;) < n. All such paths are labelled by 
n-stores. Finally, to concisely refer to the whole set of level n paths between two 
level n control states, we introduce the following notation. Let 

Q = { 1 e Qa I l(q) < n A pi — > g — -> p 2 } 



be the set of all states of A occurring on a level n path between pi and pi . If Q 

B 

is not empty, we write pi P2, where /3 is defined as: 

B= (Qb = QU{p!,p 2 }, r', 6 B = S A n (Q B x r 1 x Q B ), pi, pa)- 



Using this notation, we can also very easily translate any level 1 n-store automa- 
ton into a level n nested automaton. 



Proposition A. 2. Regular store languages are accepted by nested store au- 
tomata. 



Proof. Let A — (Q,r',S,i, /) be a level 1 automaton recognizing n-stores. 
We want to build a level n nested automaton A' = (Q' , T,S' ,i' , f) such that 
Li(A') = L(A). As no path of A labelled by a word which does not denote a 
correct store can be accepting, we may consider without loss of generality that 
the level of every state in Q is well-defined. Let Q n -i be the set of level n — 1 
states of A. The only states of level n are i and /. If n = 1, we build A' with a 
set of states Q' = Q n -i and the following set of transitions: 

5' = {i'-±+q\ i^p^q} u (s n (Q„_ x x r X Q n _ x )) 

If n > 1, for each p,q € Q n -i and £? such that p ^ q, wc first build inductively 

a nested automaton Z3' such that L\(B') = L(B). We then give .4' the following 
set of transitions: 

5' = { i' q | 3p, q e Q„-i, i -U P 4 9 } 

A A 

U {p — > q \ 3p,q e Q n -i, p q } 

U {p^f'\ 3p,q€Q n -i, P-%9~^f}- 

A store s is accepted by A' if and only if there is a path in A' labelled by 
B[...B' k from i' to /' such that s £ • • • Li(B' k )}. We thus also have 

s e . . . L(B k )], and hence s € L(A). □ 

A. 2 Reachability. 

We present here more detailed proofs of the soundness and completeness lemmas 
for Theorem 14.31 

Before proceeding, we have to present a few additional definitions and nota- 
tions. To be able to easily express and manipulate sets of possible runs of nested 
automata, we first define the notion of store expression. 

Definition A. 3. A store expression of level over alphabet r is simply a letter 
in r . A store expression of level n > is either a n-store s, the name A of 
a (nested or not) n-store automaton, a concatenation of level n store expres- 
sions, a level n — 1 store expression between square brackets [e] , or the repeated 
concatenation e + of a level n expression e. 

Also, to describe runs of nested automata we define a binary relation i — 
which expresses the choice of a particular path in a nested automaton appearing 
inside a store expression. 



Definition A. 4. Let e = uAv be a store expression where A is a nested n-store 
automaton, we write e i — ► it[u>]v whenever w € L n (A). As usual, we write 
i — > the reflexive and transitive closure of i — ►. yl sequence of store expressions 
e\. . . e m such that e\ = A, e m € S n and Vz E [1, m — 1], e.; i — ► e^+i is called 
a run of A. 

Finally, we define a concatenation operation over stores and store expressions. 

Definition A. 5. Let e = [e\e.2\, f and g be store expressions, we write e = f ■ g 
if either f = e\ and g = [e-z], or e\ = / ■ g' and g = [3^2] . Note that if e is a 
letter in r or an automaton, there are no f and g such that e = f ■ g. 

For instance, we could write [[aB][a][6cd]] = a - [[Z?][a][6cd]], or [[aS][a][fecd]] = 
[aZ?][a] • [[bed]]. Before proving the soundness of the construction of Proposition 
14.31 we need a technical lemma expressing the fact that all cycles on the ini- 
tial state of a nested automaton during the computation of (Ai) correspond to 
possible runs of the context-free process we consider. 

The following elementary lemma expresses the simple fact that if some tran- 
sition (a, pop k ) can be applied on a certain store, then it must also be applicable 
to any store with the same top-most level k — 1 store. 

Lemma A. 6. For all HCFP H. and constant store expression s, 

3t, s ■ t > t =>• Vt , s ■ t t. 

Proof. The proof is a simple induction on the size of expression s. □ 

Lemma A. 7. For all i > and nested k-store automaton B = (Q, r,S,qo,qf) 
occurring in Ai, whenever there exist a state q\ ^ qo, path labels W\ and Wi, a 
transition label C and a path q a — ^ q —> qi qt in B, then for all run 

Ai >-^> B ■ r 1 — > [ wi C W2 ] ■ r 1 — > t ■ s 

where r is any store expression, w\ 1 — ► t and [C1V2] • r — > s, we necessarily 
have s € pre^ (t ■ s) and 

Ai B ■ r [C W2 ] ■ r s. 

Proof. Let us reason by induction on i. Assume for simplicity that no transition 
leads to the initial state in any automaton occurring in A. If i — 0, then w\ = e 
and the property is trivial. Now suppose the property is true up to some rank 
i > 0. Call d the level k operation such that Ai+i = T^Ai). Consider the 
following run p of Ai+\: 

Ai+\ B ■ r [ wi C W2 } ■ r t ■ s with w\ 1— t. 



4 We say a store expression is constant when it contains no automaton. 



As wi labels a loop on the initial state of B, another possible run of Ai+\ is: 

Ai+i B ■ r [C W2 ] ■ r s. 

We only need to show that t ■ s <—* s to conclude the proof. To do this, we will 
reason by induction on the number m of new level k transitions of Ai+i (i.e. 
transitions of Ai+i not in Ai) used in the w\ cycle on qg. 

rn = 0: As w\ contains no new transition, it also labels a cycle in A4. Now, 
either transition C belongs to Ai or not. In the positive case, p is a path in 
Ai, hence the property is true by induction on i. In the case where C is a 
new transition, by definition of Ai+i, Ai admits the following run: 

Ai B ■ r [1V1UW2] ■ r h-^-> t ■ s' with w% t and [1111)2] • r ^— -> s' , 

where u is equal to s, C\ C2 or v when d is (a, pop k ), {a,push k ) or {a,push\) 
respectively By induction on i, this run verifies the property hence we have 

Ai 1 — ► B ■ r 1 — ► [ it W2 \ • r 1 — > s with t-s mj. 

By Lcmma lA.6[ this implies that Vs", t-s" > s", and in particular t-s s. 
m =>■ m + 1: Suppose the W\ cycle in B contains m + 1 new transitions. Let 
<7o — ► 9o be one of these new transitions, we have w\ = w' x Cw' 2 - Hence B 



has a path 



w 1 x> w 1 c w 2 

<7o — > qo — ► ?o — ► <?o — > qi — ► Qf 

B B B B B 



which begins with a cycle on qo labelled by w[, containing m or less new 
transitions of Ai+i- Suppose t = t% -t% and 1 — ► t\ , by induction hypothesis 
on m we have: 

Ai+\ 1 — * B ■ r [ T> w" C it)2 } ■ r t% ■ s 

and s G pre^ti ■ s). We now have to examine the way transition T> is created 
in A4+1, which depends on the type of d. As previously, by definition of Ai+\ 
there must be a run of the form 

Ai B ■ r [ u w" C1U2] ■ r t—^-> t$ ■ s, 

where u is equal to e, T>i T>2 or v when d is (a,pop k ), {a,push k ) or {a,push\) 
respectively. It is easy to show that t 3 can be chosen to be dfa)- This run 
uses a path in B starting with a cycle on go labelled by u w'{ which contains 
m or less new level k transitions: 

u to" C tu 2 

qa — ► go — > qo — ► qi — ► g/- 
8 8 b b J 

Using the induction hypothesis on m, we can now conclude that: 
Ai+i [C1V2] -r 1 — ► s and ^3 ■ s S pre^(s). 

We have t-s > to • s > <a • s s, hence t-s w s, which concludes the 

w w w 

proof. □ 



Lemma 14.51 (Soundness). Vi, L(Ai) C pre^(S). 

Proof. Assume for simplicity that no transition of an automaton occurring in A 
leads to its initial state. We reason by induction on i. The base case is trivial 
since Ao = A and L(A) C pre^(L(^4)). Now consider a store s in L(Ai+i). If s 
is accepted by Ai+i using no new transition, then it is accepted by A4. Hence by 
induction hypothesis it belongs to pre^(S). Otherwise, the accepting run must 
be of the form 

A4+1 ^> B ■ r ^-^ [ w\ C w 2 } ■ r s, 
where the path in B which generates w\ C w 2 is of the form 

W\ C W2 

with qi 7^ go- By Lemma I A. 71 there exist t,s\ such that s = t- si, t- s\ <^-* s\ and 

Ai+\ B ■ r [Cw 2 ] ■ r s\. 

Note that by definition of Td, all new transitions start from the initial states of 
automata in Ai+\. Hence, if the transition labelled by C in the previous run is 
not new, then the whole run exists in A4. By induction hypothesis on i, there 
exists s 2 £ S such that si c — > s 2 , hence by transitivity s > s%. 
If the transition labelled by C is new, then since q\ ^ qo and by definition of T4, 
d must be of the form (a,push k ) or (a,push\). Then by construction of Ai+\ 
there is a run 

A4+1 1-^ B ■ r [uw 2 ]-r ^ s 2 , 
where u is either C\ C 2 if k > 1 or v is k = 1, and s 2 can be chosen as d(si). Now 
by induction hypothesis on i, there exists S3 G S such that s 2 c — > S3, hence by 
transitivity s S3. □ 

Lemma 14.61 (Completeness). For all nested store automaton A and HCFP 
transition d, pre d (L(A) C L(Td(A)). 

Proof. Let A 1 = Td(A). Consider a store s G L(A), and let s' be any store such 
that s' £ pre dj (s). There is a run p of A recognizing s as follows: 

A^ B-n — ► -r i^U s. 

Depending on <i, we have to consider three cases: 

1. If dj = (a,pop k ), then s' = t ■ s where t is any store of level k — 1 such that 
top^t) = a, and by definition of the following run exists: 

^H%[4 fc - 1 )Ci...c I ]T^«'. 

2. If dj — (a, push k ), k > 0, then s = tt ■ r and s' = t ■ r where top^t) — a 
and i is in both L(C\) and L(C 2 ). Hence t is also accepted by the level k — 1 
automaton C\ x C 2 x -^a -1 - Thus, by definition of the following run exists: 

A' ^ [d x C 2 x .A*" 1 C 3 . . .Cj] • r ^ s'. 



3. If dj = (a, push™), then s — w ■ r and s' = a ■ r. This means C\ . . .Ci are 
level automata (i.e. letters), and C\ . . .C\ w \ = w. By definition of Td the 
following run exists: 

A' ' [aC\ w \ +1 ...Ci]-r s '. 

This establishes the fact that Td adds to the language L of its argument at least 
the set of direct predecessors of stores of L by operation d. □ 

A. 3 Constrained nested automata. 

The language of a constrained nested automaton is defined via a simple adap- 
tation of the construction of Prop. IA.1I Consider a nested automaton A = 
(Q, r, 5, i, /) of level n constrained with respect to a level 1 n-store automaton 

B = (Qb, r', 5g, is, /b) 0. First, consider the (unconstrained) nested automaton 

c c 

A' = (Q, r, S , i, /), where 5' = {p — > q I p — > (q, r) }. Second, build according 

A 

to the construction of Prop. ED a level 1 automaton A' [= (Q'i,r',5'i,i',f) 
with the same accepted language as A'. By adding to A 1 1 the control states 
of B and integrating into it the set of constrained transitions of A, one gets an 
alternating store automaton A[= (Qi, 2~", <5j, (i' A is), /'), where Q|= UQg. 
By construction, control states in Q' [ are of the form q n . . . q^ where k £ [1, n] 
and each g.j is a control state of a level k automaton occurring in A'. We define 
SI as the union of <5g and the set of all s — ► t such that: 

1. ppr — G <5'|, s = ppr, t = (pqAu), X =] and p -^-> (g, m) where C occurs 

in A and r is a control state of T>, 

2. pp ► pq' G <5'|, s = pp, t = (pq A u), X = a, and p (q, u) where C is a 

level 1 automaton occurring in A, 

3. s — ► t G 5'J, in all other cases. 

We now define the language accepted by A as the language accepted by the 
alternating automaton A{ we just defined, according to the usual notion of 
acceptance for alternating automata: L(A) = L(Ai) (please note that the initial 
state of Ai is i' A ig). 

A. 4 Constrained reachability. 

We give here three lemmas allowing to prove the correctness of the construction 
in Section [5l 

Lemma A. 8 (Termination). For all nested n-store automaton A, level 1 n- 
store automaton B and n-HCFP TL = (r,A), the sequence (Af) defined with 
respect to A and B eventually reaches T-^(A): 

3k >0, We A, T® (A® ) = A® . 

5 note that the levels of A and 3 have to be the same for L(A) to be defined. 



Proof. The algorithm for computing T^(A) is similar to the one for computing 
T-n(A), except that it labels some of the transitions of each Af by a state 
of B. As the number of such states remains unchanged throughout the whole 
computation, this does not add any unboundedness in the computation and the 
maximal number of iterations before reaching a fix-point is still finite. □ 

Lemma A.9 (Soundness). Mi, L(Af) C pre^[C]{S). 

Proof. By definition of L(Af), L(Af) C L{A t ) for all i. So, by Lemma EM we 
already have L(Af) C pre^(S). Let us reason by induction on i. By definition of 
constrained nested automata, L(Aq) = L(A) D C, hence L(Aq) C pre^[C](S). 
Now assume the property is true up to some rank i, and consider the automa- 
ton Af +1 . Note that everywhere transformation Tj 3 adds a transition in Af to 
get ^4.S_i, the alternating transitions induced in Af^xi ensure that each store 
labelling a new accepting path in the automaton is a transformation of a store 
labelling an accepting path in B. This way, one makes sure that no element of 
C in pre^(S) \ pre^ i [C](S) is added to the language of Af +1 . 

For instance, assume d = {a^push™) and some store s is accepted by ^4S_x-l 
using a a-transition newly created by Tj 3 . According to the definition of T® , 
this transition is of the form p — (<?, r), where r is a control state reachable 
in B through a path labelled by [" w. Thus, if we let w(s) = [ n aw\ for s to be 
accepted by ^.S.xJ,, then necessarily s' must be accepted by B from state r. The 
same kind of reasoning holds for the other types of operations. □ 

Lemma A. 10 (Completeness). Vi, Sf C L(^lf). 

Proof. By definition, X(^) = SnC = S c . Now suppose the property is true 
up to some rank i, and consider a store s G Sf +1 \ Sf . Let d be the operation 
such that Sf +1 = Sf U (d{Sf) n C). By definition, there is a store s' e S** 7 such 
that s' = d(s), and by induction hypothesis s' is accepted by Af. Moreover, 
since both s and s' are in C, they are accepted by B. As seen in Lemma 14.61 

transformation Td adds a new transition po — ► <7 creating in particular a path 
labelled by s. The additional constraints puts on this transition, and all paths 
in Af +1 in general, forbids any path labelled by some r using this transition to 
be accepted unless both r and d(r) also have an accepting run in B. This is the 
case for s and s', hence s G L(Af +1 ). □ 



